Academics must rebut policy-based ‘evidence’

Around the turn of the millennium, education researchers discovered evidence. They had noticed medics’ development several years earlier of something called “evidence-based practice” and concluded that education should get in on the action. What a profound moment! Tugged from its medieval insistence on superstition and liturgical dogma, education research would at last be able to embrace its own scientific awakening. Talk of “evidence” became hard to avoid, both in academic and policy circles.

This, of course, raised huge questions about what properly constituted evidence – questions that are still hotly debated. But policymakers didn’t let those questions detain them. They realised that chanting the mantra of “evidence” gave mighty potency to any argument they wished to promote.

However, my exploration of articles and speeches by politicians and their advisers, recently published in Educational Review, suggests that the term is used often with only a superficial allusion to any kind of research. Moreover, any citations are often highly selective, accompanied by unconcealed deprecation of alternative interpretations. So when politicians speak about “evidence-based policy”, what they are actually talking about is “policy-based evidence” – evidence cherry-picked to justify policies that have already been decided. And it is happening everywhere, across the spectrum of public policy.

One of the pieces I’ve examined is Dominic Cummings’ 237-page essay Some thoughts on education and political priorities, written in 2013, when he was a highly influential special adviser to then UK education secretary, Michael Gove. “Evidence” appears more than 40 times, but in most cases it is employed without any reference to specific, detailed data in support of the issue or proposition being discussed. The word “evidence” is used in lieu of actual evidence being cited.

And when a research base is claimed as evidence, it’s often misleading. For example, Cummings claims that the evidence is that “some ‘charter school’ chains/Academy chains in the US/UK have done much better than average state schools and better than state schools in the richest areas”. In fact, the most authoritative finding available at the time was the report by the Institute of Education Sciences for the US Department of Education, which showed charter schools to have had no positive impact on test scores.

I give further examples in my article of high-profile policies in education whose claimed evidence base actually flouts the consensus of academic opinion on the research corpus. Such pronouncements typically occur in forums that, while they may receive a wide airing via press releases and the media, are almost immune from academic scrutiny.

Academics must take a share of responsibility for the way that the corruption of the notion of “evidence” proceeds unimpeded. Not only do we seldom query such pronouncements, we often collude in processes that buttress them. University researchers who have undertaken contract research for large organisations or government departments will be familiar with the ubiquitous “steering groups” that offer advice and guidance in what is often, in reality, an unconcealed endeavour to influence the direction, findings and reporting of research. Why do self-respecting researchers submit to such overreach?

The answer, of course, is the pressure inside universities for staff to be earning research income. And the emphasis on “impact” in the Research Excellence Framework (REF) compounds the problem as acclaim derives from association with government departments and the putative impact of policy that originates in the research they commission. By contrast, publicly criticising government press releases or announcements will not be counted as “REFable” at all. Worse, it might be considered by academics’ employers to be dangerous.

But our responsibility as an academy is surely to question the notion of evidence when it is used, and to interrogate the validity of concepts such as “impact”. We have to engage energetically with a broad range of media regardless of whether doing so meets the official definition of REFability. We must not be shy about calling for specification of evidence, and we must be ready to provide alternative findings and interpretations.

If we don’t, politicians’ promiscuous use of terms such as “evidence-based” will sour the reputation of academia.

Gary Thomas is emeritus professor of education at the University of Birmingham. His article “Evidence, schmevidence: the abuse of the word ‘evidence’ in policy discourse about education” appears in Educational Review.